Fast wake-up in a gaze tracking system

ABSTRACT

A gaze tracking system, leaving a low power mode in response to an activation signal, captures an initial burst of eye pictures in short time by restricting the image area of a sensor, with the purpose of enabling an increased frame rate. Subsequent eye pictures are captured at normal frame rate. The first gaze point value is computed memorylessly based on the initial burst of eye pictures and no additional imagery, while subsequent values may be computed recursively by taking into account previous gaze point values or information from previous eye pictures. The restriction of the image area may be guided by a preliminary overview picture captured using the same or a different sensor. From the gaze point values, the system may derive a control signal to be supplied to a computer device with a visual display.

TECHNICAL FIELD

The invention disclosed herein generally relates to eye tracking(determination of eye position, gaze point or gaze angle) for providinginput data to a computer system. In particular, the invention provides amethod enabling fast wake-up from a sleep mode of a gaze tracking systemadapted to determine the gaze point of a viewer who watches a visualdisplay forming part of a portable or stationary personal computerdevice, a TV set, a heads-up display in a vehicle, a near-eye display ora display in a communication device with imaging and computingcapabilities, such as a mobile telephone.

BACKGROUND

Eye tracking systems forming part of human-machine interfaces (HMIs) aresubject to severe restrictions on power consumption, especially inpersonal computers and battery-powered devices. Many available eyetracking systems are programmed to enter a sleep mode after a period ofno input from a user. From the point of view of energy economy, theperiod should be as short as possible, so that the eye tracking systemwill be in a sleep mode a large percentage of the time. This must bebalanced against the desirability of high responsiveness of the system,which would seem to advocate a relatively longer period to elapse beforesleep mode is entered. This tradeoff may become less awkward, however,if efforts are directed at reducing the wakeup time of the system.

SUMMARY OF THE INVENTION

It is in view of the above considerations that the inventors have madethe present invention, which reduces or at least mitigates the problemsknown to be associated with the prior art. As such, it is an object ofthe present invention to provide eye tracking equipment with competitivepower management characteristics but low latency to user interactions atall instances when the equipment is operating, including anypower-saving or ‘sleep’ mode. A further object is to provide a gazetracking system that can be integrated in a personal computer system(e.g., desktop or laptop computer, tablet computer, notebook, net book,ultrabook, all-in-one desktop, TV, smart phone, personal digitalassistant, digital camera, heads-up display, near-eye display) withoutburdening the power performance of the computer system.

At least one of these objects is achieved by a method, computer programproduct and gaze tracking system with the features set forth in theindependent claims. The dependent claims define embodiments of theinvention.

A gaze tracking system communicates with at least one sensor with alight-sensitive surface arranged to receive light (which is reflected)from the head of a viewer of a visual display associated with a computerdevice. The at least one sensor may be a proper part of the gazetracking system or may be external to the system, e.g., may be providedas a peripheral component. The gaze tracking system is adapted tocontrol the computer device by providing a control signal to thecomputer device in accordance with the viewer's detected gaze point onthe visual display.

As used herein, a computer device may include a device with which theviewer interacts, e.g., a personal computer executing an applicationprogram which the viewer may control via a human-machine interfaceintegrated in the personal computer or provided as a peripheral device.Further, the viewer may interact with software, or a system software ormiddleware via a human-machine interface integrated in the personalcomputer or provided as a peripheral device. Furthermore, a computerdevice may refer to a software or hardware interface arranged betweenthe gaze tracking system and a personal computer executing such anapplication program. As such, the control signal may contain datarepresenting the viewer's gaze point, which the downstream computerdevice may process further to ascertain what action is to be taken.Alternatively, the control signal contains a specific computer-readablecommand derived at least in part from the gaze point, possibly in aformat suitable for interpretation by an application program.

According to an example embodiment, the gaze tracking system leaves alow power mode, e.g. a sleep mode, of the gaze tracking system inresponse to receiving a gaze tracking activation signal from one or moreinput means in the computer device. The gaze tracking system captures anoverview picture of at least a portion of the head of the viewer, or incase of multiple viewers at least a portion of the head of one of theviewers, and derives information related to the location of the eyeregion of the viewer, e.g., a bounding box, an estimated location of oneor both eyes. The gaze tracking system captures a plurality of picturesof the head of the viewer or at least a portion thereof. Doing so, thegaze tracking system reads out data at a first frame rate, inembodiments a frame rate being increased compared to a frame rate duringnormal operation, from restricted regions of the at least one sensor.The regions may be restricted in the sense that their union occupies atmost a first predetermined portion of the area of the light sensitivesurface of the at least one sensor. In embodiments of the presentinvention, the first predetermined portion is at most 10% of the area ofthe light-sensitive surface of the at least one sensor. The restrictedregions may be positioned in accordance with the eye region locationderived from the overview picture. Next, the gaze tracking systemdetermines a first value of the viewer's gaze point on the visualdisplay based on information extracted from said plurality of picturesand optionally on information from the overview picture; this operationin the gaze tracking system is independent of further imagery but maytake into account configuration settings, user-dependent andequipment-dependent calibration data and similar system constants. Basedon the first gaze point value thus determined, the gaze tracking systemprovides a first control signal to said computer device. After providingsaid first control signal, the gaze tracking system moves on tocapturing further pictures of at least a portion of the viewer's head byoperating said at least one sensor at a second frame rate, which inembodiments of the present invention is the frame rate during normaloperation. For one of said further pictures, the system determines afurther gaze point value based on information both from said furtherpicture and from at least one previous picture. Said previous picturemay be a picture in the said plurality of picture or may be one of theearlier pictures among said further pictures.

The quantity identified as frame rate in the claims may be determined asthe number of complete image frames—whether of the full sensor, asmaller portion of the sensor or of a restricted region of thesensor—that are captured per unit time. The inventors have realizedthat, in the case where an image sensor is partitioned into pixels, theframe rate is typically determined inter alia by an explicit or hiddenupper limit on the number of pixels that can be read out per unit time.Hence, if data are read out only for a restricted region, this may beperformed at a higher frame rate, which shortens the total time requiredfor the wake-up procedure.

With regard to the condition formulated as a percentage of the union ofthe restricted regions, the following is noted. A benefit of observingthis condition is that the available imaging resources (data transferrate, time, processing capability) are concentrated at a limited portionof the light-sensitive surface. If a larger area is covered byjuxtaposing (possibly partially overlapping) restricted regions, so thatthe union of the restricted regions reaches a substantially largerpercentage, then the imaging resources will not be concentrated at alimited portion of the light-sensitive surface. Instead, availableresources are spent on imaging a larger scene, which may lead to eithera longer wake-up time in the gaze tracking system or a degradation ofthe accuracy. Further, the amount 10% (=area of union of restrictedregions/area of full light-sensitive surface) is expressed primarily inrelation to a system where the full light-sensitive surface, in normalconditions (distance, etc.), provides an image size corresponding to theviewer's entire head when optimally positioned. Preferably, thelight-sensitive surface is also at least twice as wide in the horizontaldirection. It will be apparent to the skilled person that the actualpercentage occupied by the restricted regions may vary (for a given setof restricted regions) in response to a change in the light-sensitivesurface area and/or (for a given area of the light-sensitive surface) inresponse to a change in the set of restricted regions. The restrictedregions may be made even smaller—such as less than 5% or less than 3% ofthe area of the full light-sensitive region—which may shorten thewake-up process but may also require use of equipment with smallertolerances.

In one example embodiment, the first gaze point value is determined in astateless or memoryless fashion, while the further gaze point value isdetermined in a stateful or memoryful fashion. In particular, the firstgaze point value may be computed non-recursively while the further gazepoint value may be computed recursively on the basis of previouspictures of the viewer's head or on the basis of previous gaze pointvalues. It is known that recursion, iteration and similar approachesimproves accuracy by relying on previous results or previous input data.In this example embodiment, the apparent in ability to do so for thefirst gaze point determination is compensated by providing a richer setof input data, namely by enabling sensor operation at an increased framerate.

In one example embodiment, said plurality of pictures comprises threeconsecutive pictures. Optionally, a recursive or iterative computationalgorithm for determining the gaze point may take an equal number ofpictures into account, e.g., one current picture and two previouspictures. The number of pictures to be included depends on the expecteddifficulty (which may be related inter alia to the typicalsignal-to-noise ratio) in determining the gaze point in the particularapplication, as well as on the required accuracy of the gaze point. Assuch, variations of this example embodiment may include basing the gazepoint determination on a smaller or larger number of pictures of the atleast one eye of the viewer.

In one example embodiment, the first gaze point is determined by jointlyfiltering pictures in said plurality of pictures or a quantity derivedfrom said pictures, and said further gaze point is determined by jointlyfiltering said further picture, and at least one previous picture or aquantity derived from said pictures. Filtering may involve use of afinite impulse response filter or an infinite impulse response filter,particularly a recursive filter. Along the lines of the discussionabove, filtering may use a previous picture as (partial) input or mayuse a quantity derived from a previous picture, e.g., a gaze pointvalue. By joint filtering is meant an operation, which may be composedof suboperations, with the purpose of determining one gaze point value.Joint filtering does not primarily refer to an arrangement comprisingplural filtering instances carried out in parallel and yielding separatefinal results. As such, a joint filtering operation designed to providea further gaze point value and having as input a current picture and aprevious picture typically takes both these pictures into account tocompute a single further gaze point value.

In a further development of the preceding example embodiment, the gazetracking system is adapted to derive a gaze point guess from one or morepictures. In particular, the gaze point guess may be derived from asingle picture; alternatively, it is derived from several pictures, suchas all pictures in said plurality of pictures. The gaze point guess maybe derived using a numerical method known per se in the art. Based onthese, the gaze tracking system determines a gaze point value byfiltering several gaze point guesses, e.g., by averaging. Hence, even ifthe numerical method by which each gaze point guess is provided is knownto be sensitive to errors in the input data (the pictures), the gazetracking system as a whole will be less sensitive than the numericalmethod, by virtue of the subsequent filtering. In a further developmenthereof, the gaze tracking system may be configured to initiate filteringonly if the gaze point guesses remain in a bounded range. This iseffectively a condition on maximal gaze drift, to the effect thatsuccessive eye pictures that belong to a saccade rather than to afixation or smooth pursuit will not be subject to averaging. If the gazetracking system determines in this manner that there exists only onepicture of the eye in its present position (i.e., after a saccade), itmay choose to capture further pictures of the eye, until there are asufficient number of eye pictures for which the gaze point guesses aresufficiently close. Alternatively, the gaze tracking system mayexceptionally output the gaze point guess as the gaze point.Alternatively, for example embodiments where the gaze tracking systemresponds to a request for information about the viewer's gaze point atthe time of the gaze tracking activation signal, multiple gaze pointguesses on a (final portion of a) saccade following the gaze trackingactivation signal may be used to determine where the saccade came fromand thus where the viewer's gaze was located at the time of the gazetracking activation signal.

In one example embodiment, the configuration of the region to which theat least one sensor is restricted is guided by an overview picture ofthe head of the viewer. As discussed previously, the overview picturemay be used for the purpose of finding an eye region location. It is notessential what particular device or component provides the overviewpicture to the gaze tracking system, but several options are available.

Preferably, the overview picture is captured by reading out data fromthe full light-sensitive surface, or more than 50% thereof, in the atleast one sensor. For instance, the gaze tracking system may beconfigured to restrict the sensor, based on the overview picture, to theviewer's eye region, a region around one of the eyes or one regionaround each eye. To further shorten the total duration of the wake-upprocedure, the sensor may be operated at reduced (spatial) resolutionwhen it captures the overview picture, preferably in a reducedresolution mode. Because each pixel will then collect a wider beam oflight, excitation energy above a system-specific detectability thresholdwill be gathered in less time. The reduction in spatial resolution maybe achieved by binning. This may entail reading out all pixels in agroup of adjacent pixels (or every other pixel in a group of adjacentpixels) as one virtual pixel, so that excitation energy is accumulated;the adjacent pixels may be grouped in 2×2 pixel groups, 4×4 pixelgroups, groups according to rectangular patterns and the like.Alternatively or additionally, the resolution can be changed by skippingcertain pixels. For instance, it is possible to read out data only froma subset of the pixels, e.g., by reading out only every N^(th) pixel,whereby an image at a lower resolution can be collected in shorter time.

Still in the interest of shortening the wake-up procedure, the overviewpicture may be captured with active illumination. This may be achievedby activating at least one light source emitting light towards the headof the viewer. The activated light source is preferably of a wide-angletype, providing general lighting that may be expected to increase imagecontrast between different portions of the eye and the surroundingregion.

By applying resolution reduction and/or active illumination, theexposure time required to capture the overview picture may be reduced toless than half the normal exposure time in ambient lighting conditionsand full resolution. In some implementations, the exposure time may bereduced to a quarter of the normal value or even less.

As an alternative implementation, the overview picture is captured inthe form of a depth picture. The depth picture may be provided by adepth sensor, to be understood as a device for providing atwo-dimensional array of depth data of a scene. The depth sensor may beintegrated in the gaze tracking system or may be associated with thesystem, such as by the intermediary of a computer device, to which thesystem is communicatively connected. In particular, a depth sensorintended for consumer products may include an infrared or near-infraredlaser projector combined with a monochrome CMOS sensor with sensitivityin that wavelength range. Alternatively, an illuminator emitting in thevisible range is combined with a corresponding light sensor. A depthsensor may be a time-of-flight instrument measuring the time fromemission of a light pulse to receipt of its reflection. Alternatively,the depth sensor may illuminate the scene by structured light (e.g., dotpatterns, stripes) and apply triangulation methods. Alternatively, thedepth sensor includes two or more cameras and derives the depth mapstereoscopically, such as based on two or more pictures of a portion ofthe viewer's head which are captured in a short time window orsimultaneously. The depth sensor may include the at least one sensorreferred to previously; for instance, the at least one sensor may becombined with a suitable light source or may be one or both cameras(supposing at least two sensors are provided) in a stereoscopic pair. Aparticular benefit with using a depth sensor is that additionalinformation, e.g., regarding the spatial location of an eye, may bederived from the same depth picture. Indeed, because the spatial eyelocation typically varies more slowly than the eye orientation, depthpictures intended for this purpose are typically captured lessfrequently than eye pictures, in order to save resources. Availabilityof an estimate of the spatial location of the eye is useful in gazetracking according to the PCCR approach and similar techniques.

In one example embodiment aiming to further reduce the total wake-uptime, the gaze tracking activation signal triggers an interrupt in thegaze tracking system. In an implementation where the gaze trackingsystem and the computer device communicate over a bus interface with aprotocol that gives priority to certain packet types, the gaze trackingactivation signal may alternatively, but with an equivalent effect, betransmitted as a packet of the prioritized type. In particular, if a USB(Universal Serial Bus) interface is used, the gaze tracking activationsignals may be a request-type message which according to the USB is tobe handled within a period of reduced duration.

In one example embodiment, said further pictures are captured subject toa less stringent restriction on the image region, or even no restrictionat all (i.e., data are read out from the entire light-sensitive surfaceor at least a substantial portion thereof). This advantageously makesthe determination of the gaze point more robust against fast headmovements, which may cause a sudden shift of the previous eye location.Such robustness is particularly useful when the sensor is operated atnormal frame rate rather than increased frame rate. In a furtherdevelopment of this example embodiment, the region from which data areread out is enlarged and/or repositioned adaptively in dependence of adetected head movement. In particular, the region may be enlarged byperipheral padding by a number of pixels that is related to an estimatedmomentary movement velocity at the surface of the head (including bothrotary and translational motion). In particular, the region may berepositioned (to a new location on the light-sensitive area) bytranslation in a direction derived from an estimated momentary headvelocity vector. In particular, the region from which data are read outto capture said plurality of pictures (underlying the determination ofthe first gaze point value, at least in part) may be updated by purerepositioning, wherein it maintains a fixed first format (representing afirst area) but is repeatedly translated over the light-sensitivesurface in accordance with an estimated direction of motion and/or speedof motion. When pictures are captured for the purpose of determining thefurther gaze point value, a fixed second format it used, which differsfrom the first format at least in that it represents a second area,which is greater than the first area. The second area may be at least1.25 times the first area, such as 1.5 times the first area, such as1.75 times the first area, such as 2 times the first area, such as 3times the first area. Alternatively, the first and second formats maydiffer in that they comprise different numbers of pixels. In particular,the second format may comprise at least 1.25 times (such as 1.5 times,1.75 times, 2 times, 3 times) the number of pixels in the first format.

In one example embodiment, the ratio between the first frame rate, i.e.the increased frame rate (in number of frames per unit time), and thesecond frame rate, i.e. the normal frame rate, is at least 2, such as atleast 5, such as at least 10, such as at least 20, such as at least 50.The normal frame rate may be approximately 40 Hz±10 Hz, while theincreased frame rate may be 300 Hz±100 Hz.

In one example embodiment, the input means which emit the gaze trackingactivation signal is of a non-gaze type. Further preferably, the inputmeans include a body gesture sensor, an acoustic transducer, a touchdetector, a pointing device or a proximity sensor. The body gesturesensor may be optical, such as for instance a system for remote depthsensing which has gesture recognition functionality. The acoustictransducer may be combined with a speech recognition processor to form avoice-based input means. The touch detector may be optical, capacitiveor electromechanical and may be integrated in a visual display, a touchpad or a keyboard. The pointing device may be a mouse, joystick, drawingpad or the like. The proximity sensor has the functionality of producinga positive proximity indication when it establishes that a body part iscloser than a threshold distance to the sensor. The proximity sensor maybe of an optical type or a non-optical type, including capacitive.

In one example embodiment, still in the interest of reducing the wakeuptime, said plurality of images are captured using a sensor of therolling shutter type. A characteristic of a rolling shutter sensor,distinguishing it from a global shutter sensor, is that all pixels arenot imaged at a single point in time but rather by scanning across thesensor according to a predefined scanning pattern. The predefinedscanning pattern may be along horizontal or vertical lines in someorder. Unlike a rolling shutter sensor, a global shutter sensor mayattempt to approximate a snapshot photograph by recording an entireframe in a common time window. Rolling shutter sensors typically providefaster data readout, which may ideally (neglecting inter alia anyblanking lines and blanking columns in the data format) be limited onlyby the ratio of the size of the readout region and the data transferrate. This allows the gaze tracking system to begin gaze-pointprocessing of said plurality of images earlier. In particular, the gazetracking system may begin processing before all pixels in the regionhave been read out. Importantly, rolling shutter sensors typicallyprovide a finer pixel pitch than a comparable global shutter sensor(e.g., 1.1 μm vs. 3 μm). Because of the well-known unpleasantnon-synchronicity of the pixels in each image, which appears to make thesubsequent gaze point derivation more difficult, the use of a rollingshutter sensor in gaze tracking equipment is non-obvious. In thisexample embodiment however, as the inventors have realized, therestriction on the image area removes this inconvenience to a largeextent.

In one example embodiment, the wake-up procedure ends with a ready mode,which the gaze tracking system maintains for a predetermined period. Thegaze tracking system is ready to handle additional activation signals inthe ready mode. The additional activation signals are not necessarilygaze tracking activation signals but may be signals in an HMI by whichthe viewer interacts with the computer device. If no additionalactivation signal is received during the predetermined period, the gazetracking system goes back to the low power mode, e.g. the sleep mode.Similarly, if one or more additional activation signals have indeed beenreceived but the predetermined period has elapsed after the last event,the system behaves similarly. The duration of the predetermined periodmay be selected in dependence of the desired responsiveness of thecomputer device (which may be experienced as reduced if the gazetracking system introduces significant latency) and on the availablepower. In one implementation, the predetermined period may lastapproximately 30 seconds. In its ready mode, the gaze tracking systemmay operate the at least one sensor at a relatively lower frequency (orframe rate). For instance, the lower frequency may be less than ½ of thehigher frequency, preferably less than ¼, such as less than 1/10. Thiswill reduce the contribution from the at least one sensor (andassociated processing circuitry) to the total power consumption of thegaze tracking system. Still, the lower frequency may provide sufficientaccuracy (including motion accuracy) since updates of the gaze point canbe guided by previous values. In its ready mode, further, the gazetracking system may read data from the full sensor or from a regionrepresenting an area smaller than the full sensor.

In one example embodiment, there is provided a gaze tracking systemcomprising a processor configured to perform the wake-up procedureaccording to one of the implementations outlined above. The gazetracking system is associated with at least one sensor, preferably incommunicative connection with this. In particular, the at least onesensor may be selectively operable at normal frame rate, wherein datamay be read out from a relatively larger selection of regions on thelight-sensitive surface (which possibly include the full surface, butthere may alternatively be an implementation-specific upper bound on thearea of the regions), or at increased frame rate, wherein data are readout from a restricted region. The restricted region may be positioned indifferent locations on the light-sensitive surface but has a smallerarea in order to enable the sought-for increase in frame rate.

Without departing from the scope of the invention, the at least onesensor in the gaze tracking system may comprise two or more sub-sensors,wherein the normal-rate mode includes operating a first sub-sensor andthe increased-rate mode includes operating a second sub-sensor.Preferably the first sub-sensor comprises a larger light-sensitivesurface than the second sub-sensor.

In one example embodiment, there is provided a computer program productfor controlling a gaze tracking system of the type described above.

It is noted that the invention relates to all combinations of features,even if recited in mutually different claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments of the invention will now be described withreference to the accompanying drawings, on which:

FIG. 1 is a schematic drawing of a gaze tracking system arranged incooperation with a computer device equipped with a visual display;

FIG. 2 illustrates the positioning of restricted regions within alight-sensitive surface of an image sensor, with the aim of capturingpictures of a viewer's eye region efficiently;

FIG. 3 shows an implementation of a depth sensor; and

FIG. 4 is a flowchart of a method in a gaze tracking system.

All the figures are schematic and generally only show parts which arenecessary in order to elucidate the invention, whereas other parts maybe omitted or merely suggested. Unless otherwise indicated, likereference numbers on different drawings refer to structurally orfunctionally corresponding elements.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

As shown in FIG. 1, a gaze tracking system 10 according to an exampleembodiment comprises a processor 11. The processor is communicativelyconnected to a light source 13 and two imaging devices 12, 14, which maybe external to the system (and located in the same or a differentphysical device) or may be internal components in the gaze trackingsystem 10. It is envisaged inter alia that the gaze tracking system isembodied as an internal software or hardware component in a personalcomputer system (for examples, see above) which acts as interfacetowards the imaging devices 12, 14 and the light source 13 on one sideand further towards a processor in the personal computer system. Thegaze tracking system 10 may be embodied as middleware executing underthe operating system of the personal computer system. The gaze trackingsystem 10 may further be distributed, so that some tasks (e.g.,processing of pictures) are performed in software executing under theoperating system and some tasks (e.g., communicating with the lightsource 13 and the imaging devices 12, 14) in a processor 11 arranged ina peripheral unit (not shown) associated with the personal computersystem. The processor 11 may be implemented at least according to thefollowing options, or any combination of the following options:

-   -   a) One or more imaging devices (sensor(s)) are equipped with        logic with processing capabilities, which alone or jointly act        as what is referred to as the “processor” in this disclosure        including the appended claims.    -   b) A conventional standalone or integrated microprocessor is        provided as a component in the gaze tracking system (separately        from an associated computer system) in communication with the        imaging device. The microprocessor may be programmable by a low        or high level language, e.g., assembler or C, and acts as        “processor”.    -   c) An external application-specific integrated circuit (ASIC) is        provided communicatively connected to the at least one image        sensor. The logic is hardcoded as transistors in the ASIC.        Hence, the ASIC has limited processing capabilities but still        fulfils the tasks attributed to the “processor”.    -   d) The “processor” is a hard-coded integrated part of a CPU in a        computer system or is provided in the form of a hard-coded        companion chip to the CPU. The computer system may be        controllable by the gaze tracking system.    -   e) The “processor” is a software program executing in the CPU in        a computer system. The computer system may be controllable by        the gaze tracking system.    -   f) A microcontroller acts as “processor”. The microprocessor may        be of a standalone type and may alternatively be integrated in a        computer system which is controllable by the gaze tracking        system.

Via a wired or wireless, internal or external connection, the gazetracking system 10 may send and receive signals to an interface 21 in acomputer device 20, including receiving an activation signal A andsending a control signal C. If the gaze tracking system is at leastpartially integrated in the computer system 20, these signals areinternal and may be sent over internal communication interfaces, whichmay be distinguishable physically (e.g. buses, connectors, USB, TCP/IP,PCIe) or completely virtual (e.g., exceptions, events, other softwaremessages exchanged between processes executing in a memory, dataexchanged according to a predefined API syntax and the like). Thecomputer device 20 further comprises a visual display 23 and a pointingdevice 22, drawn as a mouse for exemplary purposes. A user of thecomputer device 20 may occasionally direct his or her visual attentionat the visual display 23, for instance at the example gaze point G shownon the drawing, and will be referred to as a viewer 99 in what follows.

It is pointed out that protection is sought independently for the gazetracking system 10, for the computer device 20 and for the compoundsystem shown in FIG. 1 as well, in which the gaze tracking system 10 andthe computer device 20 constitute two interconnected sections. As noted,the invention may be embodied in different combinations of processing,communication and imaging hardware or software entrusted withgaze-tracking-related tasks, regardless of their being deployed inparticular physical units; protection is sought for any suchfunctionally motivated combination as well. The functional andstructural characteristics of the gaze tracking system 10 and thecomputer device 20 according to different example embodiments have beendiscussed in previous sections of this disclosure and will be furtherexplained below.

In the deployment shown in FIG. 1, the gaze tracking system 10 isconfigured to control the computer device 20 on the basis of a detectedgaze point G, by providing a control signal C to the computer device 20.The properties of the control signal C in different example embodimentshave been discussed above. Conversely, the computer device 20 maycontrol a power state of the gaze tracking system 10 by providing anactivation signal A. If the gaze tracking system 10 is operable in a lowpower mode, e.g. a sleep mode, and a normal mode, it may respond to theactivation signal A by leaving the low power mode and entering normalmode. As a further possibility, if the gaze tracking system 10 entersthe low power mode after expiry of a timer (not shown) withpredetermined duration, the gaze tracking system 10 may respond to anactivation signal A received in the normal mode by restarting the timer.

To economize energy, the computer system 20 may be configured to issuethe activation signal A when it decides that there is a need for gazetracking input. In some implementations, a mere detected presence of theviewer may be sufficient. As discussed previously, the activation signalA may alternatively be triggered by a voice command, a body gesture orthe like. In another implementation, detected use of the pointing device22 or detected proximity to the pointing device 22 may trigger theactivation signal A. In particular, if the pointing device 22 ishand-controlled, e.g., a mouse, joystick or the like, a detected movemay trigger the activation signal A. This way, the pointer appearing onthe display may jump to the actual gaze point G of the viewer 99, whomay subsequently select a graphic control (e.g., associated with apredetermined action to be performed by the computer device 20) by amouse click or the like, possibly after fine manual adjustment of thepointer position. Similarly, if the pointing device 22 is of afinger-controlled type, such as touch pad to be actuated by one or morefingertips, the positioning of a finger may trigger the activationsignal A. In response hereto, the gaze tracking system 10 returns acontrol signal C indicating the current gaze point, to which the pointeris immediately moved (alternatively, some other visual feedback is shownto indicate detected graphical object at the viewer's gaze point that isdue to be selected), and the viewer 99 may adjust the position of thepointer by moving his or her finger(s) over the touch pad beforereleasing the finger(s) to select a graphical element in the GUIassociated with a predetermined action to be performed by the computerdevice 20. The two processes which have been described in connectionwith the hand-controlled and finger-controlled pointing devices aremethods for human-machine interaction in the compound system, for whichprotection is sought independently.

The imaging devices 12, 14 may be of different types or same type. Inother embodiments than the one shown in FIG. 1, the gaze tracking system10 may comprise a single imaging device 12; as noted above, fastacquisition of an overview picture at full or near full size is possibleinter alia if the single imaging device 12 is operable in two differentresolutions modes. At least one of the imaging devices 12, 14 isresponsible for imaging an eye region of the viewer 99 in order toprovide input data for determining a gaze point G. If the gaze trackingsystem 10 captures an overview picture of the viewer's 99 head or aportion thereof, then either the overview picture is received from anexternal imaging device (not shown) or at least one of the imagingdevices 12, 14 is configured to provide this overview picture. Theoverview picture may certainly be based on information from both imagingdevices 12, 14. As mentioned previously, the imaging devices 12, 14 maybe sensitive to visible light, to near-infrared or infrared light.Further possible configurations of the imaging devices 12, 14 and thelight source 13 include the following:

-   -   a) The imaging devices 12, 14 are two high-resolution CMOS-type        sensors, the outputs of which are processed by triangulation to        yield the depth map. Alternatively, the processing may be        simplified in that the two outputs are processed to yield a        distance to an eye or a partial depth map restricted mainly to        the eye region. The light source 13 may be used during eye        imaging, e.g., providing general illumination or a corneal        reflection. The light source 13 may alternatively be used to        shorten the exposure time of the overview picture, as explained        above.    -   b) The first imaging device 12 is a higher-resolution CMOS-type        sensor, and the second imaging device 14 is a lower-resolution        CMOS-type sensor. The first imaging device 12 provides the eye        imagery. The depth map is created by operating the second        imaging device 14 while simultaneously illuminating the viewer's        99 head by structured or coded light from the light source 13.        By way of example, FIG. 3 shows how a periodic pattern of dots        can be used to measure the depth at various points in the scene        by studying how the pattern transforms on various surfaces        (e.g., by rescaling and shearing) in dependence of the angle of        incidence.    -   c) The first imaging device 12 is a higher-resolution CMOS-type        sensor, and the second imaging device 14 is a time-of-flight        sensor. In an implementation of this type, the light source 13        is preferably modulated and synchronized with the time-of-flight        sensor.

The one or more imaging devices responsible for providing the eyeimagery may have a limited data output rate (as measured in pixels perunit time), which in turn limits the frame rate at a given image size.As discussed in other parts of this disclosure, gaze tracking algorithmswith a recursive-type approach (e.g., one or more previous eye picturescontribute as input data, together with a current eye picture, to acurrent gaze point value) may lack sufficient input data (e.g., imagerytaking the place of the previous eye picture) to complete an initialgaze point calculation at the desired accuracy. To remedy this,according to example embodiments of the invention, a burst of eyepictures are captured at increased frame rate, which is made possible byrestricting an image area. In particular, this may include reading outdata only from a restricted region (or possibly, a plurality ofrestricted regions that may differ slightly between consecutivepictures) which occupies at most 10% of the area of a light-sensitivesurface in the imaging device. For a general discussion on image arearestricting techniques, albeit unrelated to the present problem ofreducing a wake-up time, reference is made to WO 2004/45399.

To illustrate this, FIG. 2 shows the positioning of a restricted region31 in relation to a full sensor surface 30 for two different imagingdevices 12 associated with the processor 11. In the figure, therestricted regions 31 are shaped as rectangles oriented parallel to thefull sensor surface 30. This orientation may be advantageous in that therestricted regions 31 will intersect a minimal number of rows andcolumns in the sensor surface 30, which reduces the amount of blankinglines and blanking columns so that the payload occupies a greaterpercentage of the data output from each imaging device 12. The size andposition of each restricted region 31 may be guided by an overviewpicture and by searching in the overview picture for visual features(eyebrows, pupil, nose etc.) or by searching for depth patterns (noseoutline, supraorbital arches etc.) that are typically found in or nearan eye region. The result of the search in the overview picture may beoutput in the form of coordinates in the overview picture defining abounding box enclosing the eye region or a location of a reference pointin the eye region. These coordinates are then converted into equivalentcoordinates in a light-sensitive surface in the concerned imaging device12. In the situation shown in FIG. 2 shows, the conversions from acommon overview picture may have different characteristics due to theirdifferent orientations.

Referring now to FIG. 4, there is shown a method in a gaze trackingsystem for controlling a computer device according to an exampleembodiment. The method comprises leaving 401 a low power mode of thegaze tracking system in response to a gaze tracking activation signalfrom input means of the computer device, capturing 403 an overviewpicture of at least a portion of the head of the viewer and locating aneye region of the viewer in the overview picture, and capturing 405, ata first frame rate, a plurality of pictures of at least a portion of theviewer's head using the at least one sensor, from which data are readout in restricted regions, which are positioned in accordance with theeye region location and the union of which occupies at most 10% of thearea of the light-sensitive surface. The method further comprisesdetermining 407 a first gaze point value of the viewer on the visualdisplay based on information from the plurality of pictures only andoptionally on additional information from the overview picture,providing 409 a first control signal to the computer device based on thefirst gaze point value, and capturing 411, at a second frame rate, beinglower than the first frame rate, further pictures of at least a portionof the viewer's head using the at least one sensor. The method alsocomprises the steps of determining 413, for a further picture, a furthergaze point value of the viewer on the visual display based oninformation from the further picture and additionally on informationfrom at least one previous picture, and providing 415 a further controlsignal to the computer device based on the further gaze point.

Methods described in conjunction with flow charts presented herein maybe implemented in a computer-readable medium that includes instructionsfor causing a programmable processor to carry out the methods described.A “computer-readable medium” includes but is not limited to any volatileor non-volatile media, such as RAM, ROM, CD-ROM, NVRAM, EEPROM, flashmemory, and the like. The instructions may be implemented as one or moresoftware modules, which may be executed by themselves or in combinationwith other software.

There will now be briefly outlined a few examples of gaze pointdetermination algorithms with a recursive or memoryful character fordetermining a first gaze point value G1 (or a number of initial gazepoint determinations) and in subsequent gaze point values G2, G3, G4 andso forth. In these examples, the algorithms have at their disposal threeeye pictures B1, B2, B3 captured at increased frame rate during thewake-up burst and further eye pictures N1, N2, N3 captured at normalframe rate after the burst. N1 (N2) may refer to the most recent eyepicture that is available when the second (third) gaze point value G2(G3) is to be determined etc. In some examples, different algorithms maybe used to compute the first gaze point value G1 on the one hand and tocompute the subsequent gaze point values G2, G3, G4 on the other.

A first example algorithm, which uses three eye pictures to determineone gaze point, is described in terms of its input data for thedifferent gaze points in Table 1.

TABLE 1 First example algorithm Gaze point value Input data G1 B1, B2,B3 G2 B2, B3, N1 G3 B3, N1, N2 G4 N1, N2, N3The algorithm may optionally base the first gaze point value G1 on acontribution from the overview picture. Because the first gaze pointvalue G1 is computed based on eye pictures captured at increased framerate, it may be output at an early point. In order to compute thesubsequent gaze point values G2, G3, G4, it is sufficient to capturefurther eye pictures at the gaze point update frequency. It is notedthat B1, B2, B3 are nearly synchronous due to the increased frame rate;hence, even if B3 is the most recent eye picture from the burst, it maybe advisable to utilize B1 or B2 in the computation of the thirdgaze-point value G3 if any of these is of better quality than B3. Thequality may be automatically measured by one of the quality indexdescribed in the literature. The algorithm may also utilize all three ofB1, B2, B3 to determine the second and third gaze point values G2, G3,however preferably with less weight for the third than the secondconsidering their less recent dating.

A second example algorithm uses a preceding gaze point value as inputwhen it determines the subsequent gaze point values G2, G3, G4. For thefirst gaze point value G1, any preceding gaze point value will belong toa different measuring session and will not be helpful. Instead, thefirst gaze point value G1 will be computed by means of the first examplealgorithm. Table 2 illustrates the operation of the second examplealgorithm.

TABLE 2 Second example algorithm Gaze point value Input data G1 B1, B2,B3 (and optionally the overview picture) G2 G1, N1 G3 G2, N2 G4 G3, N3

A third example algorithm derives a gaze point guess from individual eyepictures. From the three eye pictures B1, B2, B3 captured at increasedframe rate during the wake-up burst, the algorithm derives gaze pointguesses g(B1), g(B2), g(B3), and from the further eye pictures N1, N2,N3 captured at normal frame rate after the burst, it derives gaze pointguesses g(N1), g(N2), g(N3). A gaze point guess may be computed by amethod known per se in the art, e.g., PCCR. The third example algorithmcomputes a gaze point value by filtering the guesses. For instance, asdescribed above, the gaze point value may be obtained by averaging.Table 3 illustrates the operation of the third example algorithm.

TABLE 3 Third example algorithm Gaze point value Input data G1 g(B1),g(B2), g(B3) G2 g(Bx), g(By), g(N1) G3 g(Bx), g(N1), g(N2) G4 g(N1),g(N2), g(N3)

Here, Bx (By) denotes that eye picture from the burst which was assignedthe highest (second highest) quality index. Optionally, the thirdexample algorithm bases the first gaze point value G1 additionally oninformation derived from the overview picture. The gaze pointdetermination may reject a current gaze point guess from the averagingif it is separated from the preceding gaze point guess by more than apredetermined threshold distance or threshold angle. This is evidence ofa saccade or a similar sudden movement. The gaze tracking system 10 maythen refrain from updating the current gaze point value and insteadcollect further eye imagery, so that a sufficient number of gaze pointguesses (e.g., 3) are available to output a new gaze point value at thedesired accuracy.

EMBODIMENTS

Advantageous embodiments of the invention further include the following:

1. A method in a gaze tracking system for controlling a computer device,said eye tracking system comprising at least one sensor arranged toreceive light from the head of a viewer, wherein:

in response to a gaze tracking activation signal from input means ofsaid computer device at said gaze tracking system, leaving a low powermode of said gaze tracking system;

capturing, at a first frame rate, a plurality of pictures of at least aportion of the viewer's head using said at least one sensor;

determining a first gaze point value of said viewer based on informationfrom said plurality of pictures;

providing a first control signal based on said first gaze point value;capturing, at a second frame rate, being lower than said first framerate, further pictures of at least a portion of the viewer's head usingsaid at least one sensor;

determining a further gaze point value of said viewer based oninformation from at least one of said further pictures; and

providing a further control signal based on said further gaze point.

2. The method of embodiment 1, further comprising, prior to thecapturing of said plurality of pictures, capturing an overview pictureof at least a portion of the head of the viewer and locating an eyeregion of the viewer in the overview picture, wherein the first gazepoint value is optionally determined based on additional informationfrom the overview picture.

3. The method of embodiment 1 or 2, wherein:

said at least one sensor includes a light-sensitive surface arranged toreceive light from the head of the viewer;

said plurality of pictures is captured by reading out data from said atleast one sensor in restricted regions, which are positioned inaccordance with the eye region location and the union of which occupiesat most 10% of the area of the light-sensitive surface; and

said further pictures are captured by reading out data from a lessrestricted or unrestricted region of the light-sensitive surface.

4. The method of any of embodiments 1 to 3, wherein:

the determination of the first gaze point is based on information fromsaid plurality of pictures and optionally on additional information fromthe overview picture; and

the determination of a further gaze point is based on information fromsaid further picture and additionally on information from at least oneprevious picture.

5. A gaze tracking system (10) comprising:

at least one sensor (12) with a light-sensitive surface arranged toreceive light from the head of a viewer, the at least one sensor beingselectively operable at a second frame rate or an a first frame rate;and

a processor (11) configured to respond to receipt, in a low power modeof the eye tracking system, of a gaze tracking activation signal (A) byperforming the following:

-   -   causing the eye tracking system to leave the low power mode;    -   capturing a plurality of pictures of at least a portion of the        viewer's head using said at least one sensor, from which data        are read out at a first frame rate;    -   determining a first gaze point value of said viewer based on        information from said plurality of pictures;    -   providing a first control signal based on said first gaze point        value;    -   capturing further pictures of at least a portion of the viewer's        head using said at least one sensor, from which data are read        out at a second frame rate, being lower than said first frame        rate;    -   determining, for each further picture, a further gaze point        value of said viewer on the basis of at least one of said        further pictures; and    -   providing a further control signal based on said further gaze        point.

6. A gaze tracking system (10) for controlling a computer device (20)associated with a visual display (23), the system comprising:

at least one sensor (12) with a light-sensitive surface arranged toreceive light from the head of a viewer of the visual display, the atleast one sensor being selectively operable to read out thelight-sensitive surface at a second frame rate or a restricted region ata first frame rate, being higher than said second frame rate; and

an overview imaging device (14) configured to capture an overviewpicture of at least a portion of the head of the viewer,

characterized by a processor (11) configured to respond to receipt, in alow power mode of the eye tracking system, of a gaze tracking activationsignal (A) from input means (22) of said computer device by performingthe following:

-   -   causing the eye tracking system to leave the low power mode;    -   capturing an overview picture of at least a portion of the head        of the viewer using said overview imaging device and locating an        eye region of the viewer in the overview picture;    -   capturing a plurality of pictures of at least a portion of the        viewer's head using said at least one sensor, from which data        are read out at a first frame rate and in restricted regions, in        which an image of an eye of the viewer is to be found and the        union of which occupies at most 10% of the area of the        light-sensitive surface;    -   determining a first gaze point value of said viewer on the        visual display based on information from said plurality of        pictures only and optionally on additional information from the        overview picture;    -   providing a first control signal to the computer device based on        said first gaze point value;    -   capturing further pictures of at least a portion of the viewer's        head using said at least one sensor, from which data are read        out at a second frame rate;    -   determining, for each further picture, a further gaze point        value of said viewer on the basis of said further picture and        additionally of at least one previous picture; and        providing a further control signal to the computer device based        on said further gaze point.

Equivalents, Extensions, Alternatives and Miscellaneous

Further embodiments of the present invention will become apparent to aperson skilled in the art after studying the description above. Eventhough the present description and drawings disclose embodiments andexamples, the invention is not restricted to these specific examples.Numerous modifications and variations can be made without departing fromthe scope of the present invention, which is defined by the accompanyingclaims. Any reference signs appearing in the claims are not to beunderstood as limiting their scope.

The systems and methods disclosed hereinabove may be implemented assoftware, firmware, hardware or a combination thereof. In a hardwareimplementation, the division of tasks between functional units referredto in the above description does not necessarily correspond to thedivision into physical units; to the contrary, one physical componentmay have multiple functionalities, and one task may be carried out byseveral physical components in cooperation. Reference is made to thediscussion above, where it was emphasized that the gaze tracking systemmay be embodied in hardware, software or middleware executing under theoperating system of a host computer system. Certain components or allcomponents may be implemented as software executed by a digital signalprocessor or microprocessor, or be implemented as hardware or as anapplication-specific integrated circuit. Such software may bedistributed on computer readable media, which may comprise computerstorage media (or non-transitory media) and communication media (ortransitory media). As is well known to a person skilled in the art, theterm computer storage media includes both volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by a computer. Further, it is well known to the skilledperson that communication media typically embodies computer readableinstructions, data structures, program modules or other data in amodulated data signal such as a carrier wave or other transportmechanism and includes any information delivery media.

1. A method in a gaze tracking system for controlling a computer deviceassociated with a visual display, said eye tracking system comprising atleast one sensor with a light-sensitive surface arranged to receivelight from the head of a viewer of the visual display, wherein: inresponse to a gaze tracking activation signal from input means of saidcomputer device at said gaze tracking system, leaving a low power modeof said gaze tracking system; capturing an overview picture of at leasta portion of the head of the viewer and locating an eye region of theviewer in the overview picture; capturing, at a first frame rate, aplurality of pictures of at least a portion of the viewer's head usingsaid at least one sensor, from which data are read out in restrictedregions, which are positioned in accordance with the eye region locationand the union of which occupies at most 10% of the area of thelight-sensitive surface; determining a first gaze point value of saidviewer on the visual display based on information from said plurality ofpictures only and optionally on additional information from the overviewpicture; providing a first control signal to the computer device basedon said first gaze point value; capturing, at a second frame rate, beinglower than said first frame rate, further pictures of at least a portionof the viewer's head using said at least one sensor; determining, for afurther picture, a further gaze point value of said viewer on the visualdisplay based on information from said further picture and additionallyon information from at least one previous picture; and providing afurther control signal to the computer device based on said further gazepoint.
 2. The method of claim 1, wherein said capturing of furtherpictures includes reading out data from a less restricted orunrestricted region of the light-sensitive surface.
 3. The method ofclaim 2, wherein said plurality of pictures are captured by reading outdata from regions of the light-sensitive surface having a first area andsaid further pictures are captured by reading out data from regions witha second area, the first area being smaller than the second.
 4. Themethod of claim 1, wherein the ratio between the first frame rate andthe second frame rate is at least 2, preferably at least 5 and mostpreferably at least
 10. 5. The method of claim 1, wherein: said firstgaze point is determined by jointly filtering pictures in said pluralityof pictures or a quantity derived from said pictures and optionallybased on additional information from the overview picture; and saidfurther gaze point is determined by jointly filtering said furtherpicture and at least one previous picture or a quantity derived fromsaid pictures.
 6. The method of claim 5, wherein: the step ofdetermining said further gaze point includes filtering gaze pointguesses derived from one or more pictures at a time; and the filteringin the step of determining said further gaze point is conditional upon amaximal drift of the gaze point guesses.
 7. The method of any claim 1,wherein the overview picture is captured by reading out data from aregion representing at least 50% of the area of the light-sensitivesurface of the at least one sensor, which at least one sensor ispreferably operated in a reduced resolution mode.
 8. The method of claim1, wherein the overview picture is captured by an imaging device whichis distinct from the at least one sensor, preferably by one of thefollowing: receiving data from a depth sensing system; receiving datafrom an RGB-type camera; receiving data from a near-infrared camera. 9.The method of claim 1, wherein the gaze tracking activation signaldirectly triggers an interrupt in the gaze tracking system, or whereinthe gaze tracking activation signal is a USB request.
 10. The method ofclaim 1, wherein the gaze tracking activation signal is received frominput means which are of a non-gaze type, and preferably of a bodygesture detector, an acoustic transducer, a touch detector, a pointingdevice or a body proximity detector type.
 11. A computer program productto be executed by a gaze tracking system for controlling a computerdevice associated with a visual display, said product comprising acomputer-readable medium with instructions for causing the gaze trackingsystem to perform the method of claim
 1. 12. A gaze tracking system forcontrolling a computer device associated with a visual display, thesystem comprising a processor and being communicatively connected: to atleast one sensor with a light-sensitive surface arranged to receivelight from the head of a viewer of the visual display, the at least onesensor being selectively operable to read out the light-sensitivesurface at a second frame rate or a restricted region at a first framerate, being higher than said first frame rate; and to an overviewimaging device configured to capture an overview picture of at least aportion of the head of the viewer, wherein the processor is configuredto respond to receipt, in a low power mode of the eye tracking system,of a gaze tracking activation signal from input means of said computerdevice by performing the following: causing the eye tracking system toleave the low power mode; capturing an overview picture of at least aportion of the head of the viewer using said overview imaging device andlocating an eye region of the viewer in the overview picture; capturinga plurality of pictures of at least a portion of the viewer's head usingsaid at least one sensor, from which data are read out at a first framerate and in restricted regions, in which an image of an eye of theviewer is to be found and the union of which occupies at most 10% of thearea of the light-sensitive surface; determining a first gaze pointvalue of said viewer on the visual display based on information fromsaid plurality of pictures only and optionally on additional informationfrom the overview picture; providing a first control signal to thecomputer device based on said first gaze point value; capturing furtherpictures of at least a portion of the viewer's head using said at leastone sensor, from which data are read out at a second frame rate;determining, for each further picture, a further gaze point value ofsaid viewer on the basis of said further picture and additionally of atleast one previous picture; and providing a further control signal tothe computer device based on said further gaze point.
 13. The gazetracking system of claim 12, further comprising a filter configured toprovide a gaze point value on the basis of a plurality of pictures ofthe viewer's head or a quantity derived from a plurality of suchpictures and optionally further on the basis of additional informationfrom the overview picture.
 14. The gaze tracking system of claim 12,wherein the overview imaging device coincides with the at least onesensor and the overview picture is preferably captured by operating theat least one sensor in a reduced resolution mode.
 15. The gaze trackingsystem of claim 12, further comprising an interrupt pin configured toreceive the gaze tracking activation signal from the input means of thecomputer device.
 16. The method of claim 2, wherein the ratio betweenthe first frame rate and the second frame rate is at least 2, preferablyat least 5 and most preferably at least
 10. 17. The method of claim 3,wherein the ratio between the first frame rate and the second frame rateis at least 2, preferably at least 5 and most preferably at least 10.18. The method of claim 2, wherein: said first gaze point is determinedby jointly filtering pictures in said plurality of pictures or aquantity derived from said pictures and optionally based on additionalinformation from the overview picture; and said further gaze point isdetermined by jointly filtering said further picture and at least oneprevious picture or a quantity derived from said pictures.
 19. Themethod of claim 3, wherein: said first gaze point is determined byjointly filtering pictures in said plurality of pictures or a quantityderived from said pictures and optionally based on additionalinformation from the overview picture; and said further gaze point isdetermined by jointly filtering said further picture and at least oneprevious picture or a quantity derived from said pictures.
 20. Themethod of claim 4, wherein: said first gaze point is determined byjointly filtering pictures in said plurality of pictures or a quantityderived from said pictures and optionally based on additionalinformation from the overview picture; and said further gaze point isdetermined by jointly filtering said further picture and at least oneprevious picture or a quantity derived from said pictures.